DLG section: Post experiment learnings & next steps

Experiment Results

After running the experiment, collect and analyse the results to determine its impact.

Key Analysis Questions:

• Which variation performed the best?
• Did the test achieve statistical significance?
• What were the key trade-offs observed?

Example Analysis Framework:

Release Decision

Based on the results, make a decision on whether to scale, iterate, or abandon the experiment.

Possible Outcomes:

• Scale: The experiment was successful with a statistically significant improvement.
• Iterate: Some improvements were seen but require further testing.
• Kill: The experiment showed no meaningful impact or negative trade-offs.

Learnings

Every experiment should provide insights, regardless of its success.

Key Learnings to Document:

• What worked and why?
• What didn’t work and why?
• Unexpected behaviours or insights from user interactions.

Next Steps

Based on the learnings, determine actionable next steps.

Actionable Next Steps Framework:

If the experiment was successful: Plan a full-scale rollout, monitor long-term performance.

If the experiment was inconclusive: Run further tests with refined hypotheses.

If the experiment was unsuccessful: Identify alternative solutions based on insights gathered.